Systems and methods for improving low dose volumetric contrast-enhanced mri

ABSTRACT

Methods and systems are provided for improving model robustness and generalizability. The method may comprise: acquiring, using a medical imaging apparatus, a medical image of a subject; reformatting the medical image of the subject in multiple scanning orientations; applying a deep network model to the medical image to improve the quality of the medical image; and outputting an improved quality image of the subject for analysis by a physician.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.17/702,468, filed Mar. 23, 2022, which is a continuation ofInternational Application No. PCT/US2020/052123, filed on Sep. 23, 2020,which claims priority to U.S. Provisional Application No. 62/905,689filed on Sep. 25, 2019, the content of which is incorporated herein inits entirety.

STATEMENT AS TO FEDERALLY SPONSORED RESEARCH

This invention was made with government support under Grant No. R44EB027560 awarded by the National Institutes of Health. The governmenthas certain rights in the invention.

BACKGROUND

Contrast agents such as Gadolinium-based contrast agents (GBCAs) havebeen used in approximately one third of Magnetic Resonance imaging (MRI)exams worldwide to create indispensable image contrast for a wide rangeof clinical applications, but pose health risks for patients with renalfailure and are known to deposit within the brain and body for patientswith normal kidney function. Recently, deep learning technique has beenused to reduce GBCA dose in volumetric contrast-enhanced MRI, butchallenges in generalizability remain due to variability in scannerhardware and clinical protocols within and across sites.

SUMMARY

The present disclosure provides improved imaging systems and methodsthat can address various drawbacks of conventional systems, includingthose recognized above. Methods and systems as described herein canimprove image quality with reduced dose level of contrast agent such asGadolinium-Based Contrast Agents (GBCAs). In particular, a generalizeddeep learning (DL) model is utilized to predict contrast-enhanced imageswith contrast dose reduction across different sites and scanners.

Traditionally, contrast agent such as Gadolinium-Based Contrast Agents(GBCAs) and others has been used in a wide range of contrast-enhancedmedical imaging such as Magnetic Resonance Imaging (MRI), or nuclearmagnetic resonance imaging, for examining pathology, predictingprognosis and evaluating treatment response for gliomas, multiplesclerosis (MS), Alzheimer's disease (AD), and the like. GBCAs are alsopervasive in other clinical applications such as evaluation of coronaryartery disease (CAD), characterization of lung masses, diagnosis ofhepatocellular carcinoma (HCC), imaging of spinal metastatic disease. In2006, an association between GBCA administration and the development ofnephrogenic systemic fibrosis (NSF) in patients with impaired renalfunction was identified. Other acute side-effects of GBCAs in subjectswith normal renal function include hypersensitivity, nausea, and chestpain. Subsequently, in 2017, U.S. FDA issued warnings and safetymeasures related to Gadolinium retention, while the regulatory bodies ofCanada, Australia and other countries issued similar warnings. Inaddition to safety advisories, the European Medicines Agency hassuspended the use of linear GBCAs. Gadolinium retention has not onlybeen reported in the CNS tissue in the form of hyper-intensities onnon-enhanced T1 W MRI, but also in other parts of the body.Environmental sustainability concerns are also being raised asgadolinium is an emerging water pollutant. Other disadvantages ofcontrast-enhanced scans include patient inconvenience during intravenousinjection, prolonged scan time, and an overall increase in imagingcosts. Even though GBCAs have a good pharmacovigilance safety profile,there is a clear need for dose reduction due to the abovementionedsafety issues and concerns. In particular, it is desirable to provide asafe imaging technique where the contrast dose can be reduced regardlessthe properties or type of the contrast materials without comprising theimaging quality or introducing additional safety issues.

Recent developments in Deep learning (DL) or machine learning (ML)techniques enable it as a potential alternative to the use of contrastdose. DL/ML has found a plethora of applications in medical imagingwhich includes denoising, super-resolution and modality conversion of,e.g., MRI to CT, T₁ to T₂. DL model has the potential to be used forgenerating contrast-enhanced images using a small fraction of thestandard dose and the pre-contrast images. Although such method may beable to reduce dose levels while maintaining non-inferior image quality,the DL enhanced images often suffer from artifacts such as streaks on areformat image (e.g., reformatted volumetric image or reconstructed 3Dimage viewed from different planes, orientations or angles).

There exists a need for providing a robust DL model that is generalizedfor (sometimes agnostic to) diverse clinical settings such as differentscanner vendors, scan protocols, patient demographics, and clinicalindications. Such a model is also desired to produce artifact-freeimages and support a variety of clinical use cases such as multiplanarreformat (MPR) for oblique visualizations of 3D images, thus enablingthe model to be deployed and integrated within a standard clinicalworkflow.

Systems and methods described herein can address the abovementioneddrawbacks of the conventional solutions. In particular, the providedsystems and methods may involve a DL model including a unique set ofalgorithms and methods that improve the model robustness andgeneralizability. The algorithms and methods may include, for example,multi-planar reconstruction, 2.5D deep learning model,enhancement-weighted L1, perceptual and adversarial losses algorithmsand methods, as well as pre-processing algorithms that are used topre-process the input pre-contrast and low-dose images prior to themodel predicting the corresponding contrast-enhanced images.

In an aspect, a method is provided for computer-implemented method forimproving image quality with reduced dose of contrast agent. The methodcomprises: acquiring, using a medical imaging apparatus, a medical imageof a subject with a reduced dose of contrast agent; reformatting themedical image of the subject in multiple orientations to generate aplurality of reformat medical images; and applying a deep network modelto the plurality of reformat medical images to generate a predictedmedical image with improved quality.

In a related yet separated aspect, a non-transitory computer-readablestorage medium including instructions that, when executed by one or moreprocessors, cause the one or more processors to perform operations. Theoperations comprise: acquiring, using a medical imaging apparatus, amedical image of a subject with a reduced dose of contrast agent;reformatting the medical image of the subject in multiple orientationsto generate a plurality of reformat medical images; and applying a deepnetwork model to the plurality of reformat medical images to generate apredicted medical image with improved quality.

In some embodiments, the medical imaging apparatus is a transformingmagnetic resonance (MR) device. In some embodiments, the medical imageis a 2.5D volumetric image.

In some embodiments, the multiple orientations include at least oneorientation that is not in the direction of the scanning plane. In someembodiments, the method or the operations further comprise rotating eachof the plurality of reformat medical images into various angles togenerate a plurality of rotated reformat medical images. In some cases,the deep network model is applied to the plurality of rotated reformatmedical images to output a plurality of predicted images. The pluralityof predicted images as an output of the deep network model are rotatedto be aligned to a scanning plane. In some instances, the method or theoperations further comprise averaging the plurality of predicted imagesafter rotated to be aligned to the scanning plane to generate thepredicted medical image with improved quality. In some embodiments, thepredicted medical image with improved quality is obtained by averaging aplurality of predicted medical images corresponding to the plurality ofthe reformat medical images.

Additionally, methods and systems of the present disclosure may beapplied to existing systems without a need of a change of the underlyinginfrastructure. In particular, the provided methods and systems mayreduce the dose level of contrast agent at no additional cost ofhardware component and can be deployed regardless of the configurationor specification of the underlying infrastructure.

Additional aspects and advantages of the present disclosure will becomereadily apparent to those skilled in this art from the followingdetailed description, wherein only illustrative embodiments of thepresent disclosure are shown and described. As will be realized, thepresent disclosure is capable of other and different embodiments, andits several details are capable of modifications in various obviousrespects, all without departing from the disclosure. Accordingly, thedrawings and descriptions are to be regarded as illustrative in nature,and not as restrictive

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.To the extent publications and patents or patent applicationsincorporated by reference contradict the disclosure contained in thespecification, the specification is intended to supersede and/or takeprecedence over any such contradictory material.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings (also “Figure” and “FIG.” herein), of which:

FIG. 1 shows an example of a workflow for processing and reconstructingmagnetic resonance imaging (MRI) volumetric image data.

FIG. 2 shows an example of data collected from the two different sites.

FIG. 3 shows the analytic results of a study.

FIG. 4 schematically illustrates a magnetic resonance imaging (MRI)system in which an imaging enhancer of the presenting disclosure may beimplemented.

FIG. 5 shows an example of a scan procedure or scanning protocolutilized for collecting the experiment data in the study.

FIG. 6 illustrates an example of a reformat MPR reconstructed image thathave a quality improved over the reformat MRI image generated using theconventional method.

FIG. 7 shows an example of a pre-processing method, in accordance withsome embodiments herein.

FIG. 8 shows an example of a U-Net style encoder-decoder networkarchitecture, in accordance with some embodiments herein.

FIG. 9 shows an example of the discriminator, in accordance with someembodiments herein.

FIG. 10 shows an experiment including data distribution andheterogeneity of a study dataset from three institutions, threedifferent manufacturers, and eight different scanner models.

FIG. 11 schematically illustrates systems and methods that are utilizedto monotonically improve the image quality.

FIG. 12 shows examples of pre-contrast, low-dose, full-dose ground truthimage data and synthesized images along with the quantitative metricsfor cases from different sites and scanners.

FIG. 13 shows examples illustrating effect of the number of rotationangles in MPR on the quality of the output image and processing time.

DETAILED DESCRIPTION

While various embodiments of the invention have been shown and describedherein, it will be obvious to those skilled in the art that suchembodiments are provided by way of example only. Numerous variations,changes, and substitutions may occur to those skilled in the art withoutdeparting from the invention. It should be understood that variousalternatives to the embodiments of the invention described herein may beemployed.

Gadolinium-based contrast agents (GBCAs) are widely used in magneticresonance imaging (MRI) exams and have been indispensable for monitoringtreatment and investigating pathology in myriad applications includingangiography, multiple sclerosis and tumor detection. Recently, theidentification of prolonged gadolinium deposition within the brain andbody has raised safety concerns about the usage of GBCAs. Reducing theGBCA dose reduces the degree of deposition, but also degrades contrastenhancement and tumor conspicuity. A reduced dose exam that retainscontrast enhancement is therefore greatly relevant for patients who needrepeated contrast administration (e.g., multiple sclerosis patients) andare at high risk of gadolinium deposition (e.g., children).

Though MRI, Gadolinium-based contrast agents, MRI data examples areprimarily provided herein, it should be understood that the presentapproach can be used in other imaging modality contexts and/or othercontrast-enhanced imaging. For instance, the presently describedapproach may be employed on data acquired by other types of tomographicscanners including, but not limited to, computed tomography (CT), singlephoton emission computed tomography (SPECT) scanners, Positron EmissionTomography (PET), functional magnetic resonance imaging (fMRI), orvarious other types of imaging scanners or techniques wherein a contrastagent may be utilized for enhancing the contrast.

Deep learning (DL) framework has been used to reduce GBCA dose levelswhile maintaining image quality and contrast enhancement for volumetricMRI. As an example, a DL model may use a U-net encoder-decoderarchitecture to enhance the image contrast from a low-dose contrastimage. However, the conventional DL models may only work well with scansfrom a single clinical site without considering generalizability todifferent sites with different clinical workflows. Moreover, theconventional DL models may evaluate image quality for individual 2Dslices in the 3D volume, even though clinicians frequently requirevolumetric images to visualize complex 3D enhancing structures such asblood vessels and tumors from various angles or orientations.

The present disclosure provides systems and methods that can addressvarious drawbacks of conventional systems, including those recognizedabove. Methods and systems of the presenting disclosure capable ofimproving model robustness and deployment in real clinical settings. Forinstance, the provided methods and systems are capable of adapting todifferent clinical sites, each with different MRI scanner hardware andimaging protocols. In addition, the provided methods and systems mayprovide improved performance while retaining multi-planar reformat (MPR)capability to maintain the clinician workflow and enable obliquevisualizations of the complex enhancing microstructure.

Methods and systems herein may provide enhancements to the DL model totackle real-world variability in clinical settings. The DL model istrained and tested on patient scans from different hospitals acrossdifferent MRI platforms with different scanning planes, scan times, andresolutions, and with different mechanisms for administering GBCA. Therobustness of the DL models may be improved in these settings withimproved generalizability across a heterogeneity of data.

Multi-Planar Reformat (MPR)

In a conventional DL pipeline, 2D slices from the 3D volume may beseparately processed and trained with standard 2D data augmentation(e.g. rotations and flips). The choice of a 2D model is often motivatedby memory limitations during training, and performance requirementsduring inference. In some cases, DL framework may process the data in a“2.5D” manner, in which multiple adjacent slices are input to a networkand the central slice is predicted. However, both 2D and 2.5D processingmay neglect the true volumetric nature of the acquisition. As the 3Dvolume is typically reformatted into arbitrary planes during theclinical workflow (e.g., oblique view, views from orientations/anglesthat are oblique to the scanning plane/orientation), and sites may use adifferent scanning orientation as part of their MRI protocol, 2Dprocessing can lead to images with streaking artifacts in the reformatvolumetric images (e.g., reformat into planes that are orthogonal to thescanning plane).

Methods and systems described herein may beneficially eliminate theartifacts (e.g., streaking artifacts) in reformat images therebyenhancing the image quality with reduced contrast dose. As describedabove, reformatting a 3D volume image to view the image in multipleplanes (e.g., orthogonal or oblique planes) is common in a standardclinical workflow. In some cases, though training a model to enhance the2.5D image may reduce the streaking artifacts in the plane ofacquisition, reformatting to other orientations may still causestreaking artifacts. Methods and systems as described herein may enableartifact-free visualizations in any selected plane or viewing direction(e.g., oblique view). Additionally, the model may be trained to learnintricate or complex 3D enhancing structures such as blood vessels ortumors.

FIG. 1 shows an example of a workflow for processing and reconstructingMRI volumetric image data. As illustrated in the example, the inputimage 110 may be image slices that are acquired without contrast agent(e.g., pre-contrast image slice 101) and/or with reduced contrast dose(e.g., low-dose image slice 103). In some cases, the raw input image maybe 2D image slices. A deep learning (DL) model such as a U-netencoder-Decoder 111 model may be used to predict an inference result112. While the DL model 111 may be a 2D model that is trained togenerate an enhanced image within each slice, it may produceinconsistent image enhancement across slices such as streaking artifactsin image reformats. For instance, when the inference result isreformatted 113 to generate a reformat image in the orthogonal direction114, because the input 2D image 110 matches the scanning plane, thereformat image 114 may contain reformat artifacts such as streakingartifacts in the orthogonal directions.

Such reformat artifacts may be alleviated by adopting a multi-planarreformat (MPR) method 120 and using a 2.5D trained model 131. The MPRmethod may beneficially augment the input volumetric data in multipleorientations. As shown in FIG. 1 , a selected number of input slices ofthe pre-contrast or low-dose images 110 may be stacked channel-wise tocreate a 2.5D volumetric input image. The number of input slices forforming the 2.5D volumetric input image can be any number such as atleast two, three, four, five, six, seven, eight, nine, ten slices may bestacked. In some cases, the number of input slices may be determinedbased on the physiologically or biochemically important structures inregions of interest such as microstructures where a volumetric imagewithout artifacts are highly desired. For instance, the number of inputslices may be selected such that microstructure (e.g., blood vessels ortumors) may be mostly contained in the input 2.5D volumetric image.Alternatively or additionally, the number of slices may be determinedbased on empirical data or selected by a user. In some cases, the numberof slices may be optimized according the computational power and/ormemory storage of the computing system.

Next, the input 2.5D volumetric image may be reformatted into multipleaxes such as principal axes (e.g., sagittal, coronal, and axial) togenerate multiple reformatted volumetric images 121. The multipleorientations for reformatting the 2.5D volumetric images may be in anysuitable directions that need not be aligned to the principal axes.Additionally, the number of orientations for reformatting the volumetricimages can be any number greater than one, two, three, four, five andthe like so long as at least one of the multiple reformatted volumetricimages is along an orientation that is oblique to or orthogonal to thescanning plane.

At inference stage, each of the multiple reformatted volumetric imagesmay be rotated by a series of angles to produce a plurality of rotatedreformat volumetric images 122 thereby further augmenting the inputdata. For example, each of the three reformatted volumetric images 121(e.g., sagittal, coronal, and axial) may be rotated by five equispacedangles between 0-90° resulting in 15 volumetric images 122. It should benoted that the angle step and the angle range can be in any suitablerange. For example, the angle step may not be a constant and the numberof rotational angles can vary based on different applications, cases, ordeployment scenarios. In another example, the volumetric images can berotated across any angle range that is greater than, smaller than orpartially overlapping with 0-90°. The effect of the number of therotational angles on the predicted MPR images are described laterherein.

The plurality of rotated volumetric 2.5D images 122 may then be fed tothe 2.5D trained model 131 for inference. The output of the 2.5D trainedmodel includes a plurality of contrast-enhanced 2.5 D volumetric images.In some cases, the final inference result 132, which is referred to asthe “MPR reconstruction”, may be an average of the plurality ofcontrast-enhanced 2.5 D volumetric images after rotating back to theoriginal acquisition/scanning plane. For instance, the 15 enhanced 2.5 Dvolumetric images may be rotated back to be aligned to the scanningplane and the mean of such volumetric images is the MPR reconstructionor the final inference result 132. The plurality of predicted 2.5 Dvolumetric images may be rotated to be aligned to the original scanningplane or the same orientation such that an average of the plurality of2.5D volumetric images may be computed. The plurality of enhanced 2.5Dvolumetric images may be rotated to be aligned to the same directionthat may or may not be in the original scanning plane. The MPRreconstruction method beneficially allows to add a 3D context to thenetwork while benefitting from the performance gains of 2D processing.

As illustrated in FIG. 1 , when the MPR reconstruction image 132 isreformatted 133 into a plane orthogonal to the original acquisitionplane, the reformat image 135 does not present streaking artifacts. Thequality of the predicted MPR reconstruction image may be quantified byquantitative image quality metrics such as peak signal to noise ratio(PSNR), and structural similarity (SSIM). The image quality metrics arecalculated for the conventional model 111 and the presented model 131,and an example of the result showing the quality of the reformat images114, 135 and ground truth 140 are illustrated in FIG. 3 .

Data Collection

In an example, under IRB approval and patient consent, the scanningprotocol was implemented in two sites. FIG. 2 shows the example of datacollected from the two sites. 24 patients (16 training, 8 testing) wererecruited from Site 1 and 28 (23 training, 5 testing) from Site 2.Differences between scanner hardware and protocol are highlighted inTable 1. In particular, the two sites used different scanner hardware,and had great variability in scanning protocol. Notably, Site 1 usedpower injection to administer GBCA, while Site 2 used manual injection,leading to differences in enhancement time and strength.

As an example of collecting data for training the model, multiple scanswith reduced dose level as well as a full-dose scan may be performed.The multiple scans with reduced dose level may include, for example, alow-dose (e.g., 10%) contrast-enhanced MRI and a pre-contrast (e.g.,zero contrast) may be performed. For instance, for each participant, two3D T₁-weighted images were obtained: pre-contrast and post-10% dosecontrast (0.01 mmol/kg). For training and clinical validation, theremaining 90% of the standard contrast dose (full-dose equivalent,100%-dose) was administrated and a third 3D T₁-weighted image(100%-dose) was obtained. Signal normalization is performed to removesystematic differences (e.g., transmit and receive gains) that may havecaused signal intensity changes between different acquisitions acrossdifferent scanner platforms and hospital sites. Then, nonlinear affineco-registration between pre-dose, 10%-dose, and 100%-dose images areperformed. The DL model used a U-Net encoder-decoder architecture, withthe underlying assumption that the contrast-related signal betweenpre-contrast and low-dose contrast-enhanced images was nonlinearlyscaled to the full-dose contrast images. Additionally, images from othercontrasts such as T₂ and T₂-FLAIR can be included as part of the inputto improve the model prediction.

FIG. 5 shows an example of a scan procedure or scanning protocol 500utilized for collecting data for the studies or experiments shown inFIGS. 2, 3, and 10-12 . In the illustrated scan protocol, each patientunderwent three scans in a single imaging session. Scan 1 waspre-contrast 3D T₁-weighted MRI, followed by Scan 2 with 10% of thestandard dose of 0.1 mmol/kg. Images from Scan 1 and 2 were used asinput to the DL network. Ground truth images were obtained from Scan 3,after administering the remaining 90% of the contrast dose (i.e., fulldose).

During inference, after deployment of the provided systems, only onescan without contrast agent (e.g., similar to scan 1), or a scan withreduced contrast dose (e.g., similar to scan 2) may be performed. Suchinput image data may then be processed by the trained model to output apredicted MPR reconstructed image with enhanced contrast. In some cases,after deploying the model to a clinical site, a user (e.g., physician)may be permitted to choose a reduced dose level that can be any level inthe range from 0 to 30% for acquiring the medical image data. It shouldbe noted that depending on the practical implementation and user desireddose reduction level, the reduced dose level can be any number in arange greater than 30%.

Inter-Site Generalizability

The conventional model may be limited by evaluating patients from asingle site with identical scanning protocol. In real clinical settings,each site may tailor its protocol based on the capabilities of thescanner hardware and standard procedures. For example, a model trainedon Site 2 may perform poorly on cases from Site 1 (FIG. 2 , middle).

The provided DL model may have improved generalizability. The DL modelmay be trained with a proprietary training pipeline. For example, thetraining pipeline may comprise first scaling each image to a nominalresolution of 1 mm³ and in-plane matrix size of 256×256, followed byapplying the MPR processing. As the DL model is fully convolutional,inference can be run at the native resolution of the acquisition withoutresampling.

Based on the qualitative and quantitative results, the addition of MPRprocessing, resolution re-sampling, and inter-site training led to greatimprovement in model robustness and generalizability. In optionalembodiments, the model may be a full 3D model. For instance, the modelmay be a 3D patch-based model that may alleviate both MPR processing,and memory usage. The provided training methods and model framework maybe applied to different sites with different scanner platforms, and/oracross different MRI vendors.

Network Architecture and Processes

FIG. 6 schematically illustrates another example of an MPR reconstructedimage 624 that have improved quality compared to the MRI image predictedusing the conventional method 611. The workflow 600 for processing andreconstructing MRI volumetric image data 623 and the reformat MPRreconstructed image 624 can be the same as those as described in FIG. 1. For example, the input image 610 may include a plurality of 2D imageslices that are acquired without contrast agent (e.g., pre-contrastimage slice) and/or with reduced contrast dose (e.g., low-dose imageslice). The input images may be acquired in a scanning plane (e.g.,axial) or along a scanning orientation. A selected number of the imageslices are stacked to form a 2.5D volumetric input image which isfurther processed using the multiplanar reconstruction (MPR) method 620as described above.

For example, the input 2.5D volumetric image may be reformatted intomultiple axes such as principal axes (e.g., sagittal, coronal, andaxial) to generate multiple reformatted volumetric images (e.g., SAG,AX, COR). It should be noted that the 2.5D volumetric image can bereformatted into any orientations that may or may not be aligned withthe principal axes.

Each of the multiple reformatted volumetric images may be rotated by aseries of angles to produce a plurality of rotated reformat images. Forexample, each of the three reformatted volumetric images (e.g.,sagittal, coronal, and axial) may be rotated by five angles between0-90° resulting in 15 rotated reformat volumetric images. The multiplereformatted volumetric images (e.g., sagittal, coronal, and axial) mayor may not be rotated at the same angle or rotated into the same numberof orientations.

The plurality of rotated volumetric images 122 may then be processed bythe trained model 621 to produce a plurality of enhanced volumetricimages. In some cases, the MPR reconstruction image 623 or the inferenceresult image is the average of the plurality of inference volumes afterrotating back to the original plane of acquisition. The MPRreconstruction image when is reformatted to be viewed at a selectedorientation (e.g., orthogonal/oblique to the scanning plane), thereformat image 624 may not contain streaking artifacts compared to thereformat image obtained using the single inference method 611 and/or thesingle inference model.

Network Architecture and Data Processing

Using the multiplanar reconstruction (MPR) technique, the deep learningmodel may be trained with volumetric images (e.g., augmented 2.5Dimages) such as from the multiple orientations (e.g., three principalaxes). The model may be a trained deep learning model for enhancing thequality of volumetric MRI images acquired using reduced contrast dose.In some embodiments, the model may include an artificial neural networkthat can employ any type of neural network model, such as a feedforwardneural network, radial basis function network, recurrent neural network,convolutional neural network, deep residual learning network and thelike. In some embodiments, the machine learning algorithm may comprise adeep learning algorithm such as convolutional neural network (CNN).Examples of machine learning algorithms may include a support vectormachine (SVM), a naïve Bayes classification, a random forest, a deeplearning model such as neural network, or other supervised learningalgorithm or unsupervised learning algorithm. The model network may be adeep learning network such as CNN that may comprise multiple layers. Forexample, the CNN model may comprise at least an input layer, a number ofhidden layers and an output layer. A CNN model may comprise any totalnumber of layers, and any number of hidden layers. The simplestarchitecture of a neural network starts with an input layer followed bya sequence of intermediate or hidden layers, and ends with output layer.The hidden or intermediate layers may act as learnable featureextractors, while the output layer in this example provides 2.5Dvolumetric images with enhanced quality (e.g., enhanced contrast). Eachlayer of the neural network may comprise a number of neurons (or nodes).A neuron receives input that comes either directly from the input data(e.g., low quality image data, image data acquired with reduced contrastdose, etc.) or the output of other neurons, and performs a specificoperation, e.g., summation. In some cases, a connection from an input toa neuron is associated with a weight (or weighting factor). In somecases, the neuron may sum up the products of all pairs of inputs andtheir associated weights. In some cases, the weighted sum is offset witha bias. In some cases, the output of a neuron may be gated using athreshold or activation function. The activation function may be linearor non-linear. The activation function may be, for example, a rectifiedlinear unit (ReLU) activation function or other functions such assaturating hyperbolic tangent, identity, binary step, logistic, arcTan,softsign, parameteric rectified linear unit, exponential linear unit,softPlus, bent identity, softExponential, Sinusoid, Sinc, Gaussian,sigmoid functions, or any combination thereof.

In some embodiments, the network may be an encoder-decoder network or aU-net encoder-decoder network. A U-net is an auto-encoder in which theoutputs from the encoder-half of the network are concatenated with themirrored counterparts in the decoder-half of the network. The U-net mayreplace pooling operations by upsampling operators thereby increasingthe resolution of the output.

In some embodiments, the model for enhancing the volumetric imagequality may be trained using supervised learning. For example, in orderto train the deep learning network, pairs of pre-contrast and low-doseimages as input and the full-dose image as the ground truth frommultiple subjects, scanners, clinical sites or databases may be providedas training dataset.

In some cases, the input datasets may be pre-processed prior to trainingor inference. FIG. 7 shows an example of a pre-processing method 700, inaccordance with some embodiments herein. As shown in the example, theinput data including the raw pre-contrast, low-dose, and full-dose image(i.e., ground truth) may be sequentially preprocessed to generatepreprocessed image data 710. The raw image data may be received from astandard clinical workflow, as a DICOM-based software application orother imaging software applications. As an example, the input data 701may be acquired using a scan protocol as described in FIG. 5 . Forinstance, three scans including a first scan with zero contrast dose, asecond scan with a reduced dose level and a third scan with full dosemay be operated. The reduced dose image data used for training themodel, however, can include images acquired at various reduced doselevel such as no more than 1%, 5%, 10%, 15%, 20%, any number higher than20% or lower than 1%, or any number in-between. For example, the inputdata may include image data acquired from two scans including a fulldose scan as ground truth data and a paired scan at a reduced level(e.g., zero dose or any level as described above). Alternatively, theinput data may be acquired using more than three scans with multiplescans at different levels of contrast dose. Additionally, the input datamay comprise augmented datasets obtained from simulation. For instance,image data from clinical database may be used to generate low qualityimage data mimicking the image data acquired with reduced contrast dose.In an example, artifacts may be added to raw image data to mimic imagedata reconstructed from images acquired with reduced contrast dose.

In the illustrated example, pro-processing algorithm such asskull-stripping 703 may be performed to isolate the brain image fromcranial or non-brain tissues by eliminating signals from extra-cranialand non-brain tissues using the DL-based library. Based on the tissues,organs and use application, other suitable preprocessing algorithms maybe adopted to improve the processing speed and accuracy of diagnosis. Insome cases, to account for patient movement between the three scans, thelow-dose and full-dose images may be co-registered to the pre-contrastimage 705. In some cases, given that the transmit and receive gains mayvary for different acquisitions, signal normalization may be performedthrough histogram equalization 707. Relative intensity scaling may beperformed between the pre-contrast, low-dose, and full-dose forintra-scan image normalization. As the multi-institutional datasetinclude images with different voxel and matrix sizes, the 3D volume maybe interpolated to an isotropic resolution of 0.5 mm³ and whereverapplicable, zero-padded images at each slice to a dimension of 512×512.The image data may have sufficiently high resolution to enable the DLnetwork to learn small enhancing structures, such as lesions andmetastases. In some cases, scaling and registration parameters may beestimated on the skull-stripped images and then applied to the originalimages 709. The preprocessing parameters estimated from theskull-stripped brain may be applied to the original images to obtain thepreprocessed image volumes 710.

Next, the preprocessed image data 710 is used to train anencoder-decoder network to reconstruct the contrast-enhanced image. Thenetwork may be trained with an assumption that the contrast signal inthe full-dose is a non-linearly scaled version of the noisy contrastuptake between the low-dose and the pre-contrast images. The model maynot explicitly require the difference image between low-dose andpre-contrast.

FIG. 8 shows an example of a U-Net style encoder-decoder networkarchitecture 800, in accordance with some embodiments herein. In theillustrated example, each encoder block has three 2D convolution layers(3×3) with ReLU followed by a maxpool (2×2) to downsample the featurespace by a factor of two. The decoder blocks have a similar structurewith maxpool replaced with upsample layers. To restore spatialinformation lost during downsampling and prevent resolution loss,decoder layers are concatenated with features of the correspondingencoder layer using skip connections. The network may be trained with acombination of L1 (mean absolute error) and structural similarity index(SSIM) losses. Such U-Net style encoder-decoder network architecture maybe capable of producing a linear 10× scaling of the contrast uptakebetween low-dose and zero-dose, without picking up noise along with theenhancement signal.

As shown in FIG. 8 , the input data to the network may be a plurality ofaugmented volumetric images generated using the MPR method as describedabove. In the example, seven slices each of pre-contrast and low-doseimages are stacked channel-wise to create a 14-channel input volumetricdata for training the model to predict the central full-dose slices 803.

Enhancement and Weighted L1 Loss

In some situations, even after signal normalization and scaling isapplied, the difference between the low-dose and pre-contrast images mayhave enhancement-like noise perturbations which may mislead training ofthe network. To make the network pay more attention to the actualenhancement regions, the L1 loss may be weighted with an enhancementmask. The mask is continuous in nature and is computed from theskull-stripped difference between low-dose and pre-contrast images,normalized between 0 and 1. The enhancement mask can be considered as anormalized smooth version of the contrast uptake.

Perceptual and Adversarial Losses

It is desirable to train the network to focus on the structuralinformation in the areas of enhancement as well as high frequency andtexture details which are crucial for making confident diagnosticdecisions. A simple combination of L1 and structural similarity index(SSIM) losses may tend to suppress high-frequency signal information andthe obtained results may have a smoother appearance, which is perceivedas a loss of image resolution. To address this issue, a perceptual lossfrom a convolutional network (e.g., VGG-19 network consisting of 19layers including 6 convolution layers, 3 Fully connected layer, 5MaxPool layers and 1 SoftMax layer which is pre-trained on ImageNetdataset) is employed. The perceptual loss is effective in style-transferand super-resolution tasks. For example, the perceptual loss can becomputed from the third convolution layer of the third block (e.g.,block3 conv3) of a VGG-19 network, by taking the mean squared error(MSE) of the layer activations on the ground truth and prediction.

In some cases, to further improve the overall perceptual quality, anadversarial loss is introduced through a discriminator, trained inparallel to the encoder-decoder network, to predict whether thegenerated image is real or fake. FIG. 9 shows an example of thediscriminator 900, in accordance with some embodiments herein. Thediscriminator 900 has a series of spectral normalized convolution layerswith Leaky ReLU activations and predicts a 32×32 patch. Unlike aconventional discriminator, which predicts a binary value (e.g., 0 forfake and 1 for real), the “patch discriminator” 900 predicts a matrix ofprobabilities which helps in the stability of the training process andfaster convergence. The spectral normalized convolution layer employs aweight normalization technique to further stabilize discriminatortraining. The patch discriminator, as shown in FIG. 9 , can be trainedwith MSE loss, and Gaussian noise may be added to the inputs for smoothconvergence.

The function for configuring the network model can be formulated asbelow:

G*=argmin_(G)[λGANL_(GAN)(G)+λ_(L1) L _(L1)(M _(enh) G)+λ_(SSIM) L_(SSIM)(G)+λ_(VGG) L _(VGG)(G)]

where M_(enh) is the enhancement mask and the adversarial loss L_(GAN)can be written as L_(GAN)=_(maxD)L_(GAN)(G,D), where G is the U-Netgenerator and D is the patch-discriminator. The loss weights λ_(L),λ_(SSIM), λ_(VGG) and λ_(GAN) can be determined empirically. With theabovementioned processes and methods, a single model is trained to makeaccurate predictions on images from various institutions and scanners.

EXAMPLE

FIG. 3 shows an example of analytic results of a study to evaluate thegeneralizability and accuracy of the provided model. In the illustratedexample, the results show comparison of ground-truth (left), originalmodel (middle), and proposed model (right) inference result on a testcase from Site 1 (red arrow shows lesion conspicuity). The conventionalmodel was trained on data from Site 2 only. This example is consistentwith the MRI scanning data illustrated in FIG. 2 . The provided modelwas trained on data from both sites, and used MPR processing andresolution resampling. In this study, the result qualitatively shows theeffect of MPR processing on one example from the test set. By averagingthe result of many MPR reconstructions, streaking artifacts thatmanifest as false enhancement are suppressed. As shown in FIG. 3 , oneslice of a ground-truth contrast-enhanced image (left) is compared tothe inference results from the model trained on Site 2 (middle) and themodel trained on Sites 1 and 2 simultaneously (right). By accounting fordifferences in resolution and other protocol deviations, the providedmodel demonstrates qualitative improvement in generalizability.Quantitative image quality metrics such as peak signal to noise ratio(PSNR), and structural similarity (SSIM) were calculated for all theconventional model and the presented model. The average PSNR and SSIM onthe test set for the conventional and presented model was 32.81 dB(38.12 dB) and 0.872 (0.951), respectively. Better image quality may beachieved using the methods and systems in the present disclosure.

In the study as illustrated in FIG. 3 , a deep learning (DL) frameworkas described elsewhere herein is applied for low-dose (e.g., 10%)contrast-enhanced MRI. For each participant, two 3D T₁-weighted imageswere obtained: pre-contrast and post-10% dose contrast (0.01 mmol/kg).For training and clinical validation, the remaining 90% of the standardcontrast dose (full-dose equivalent, 100%-dose) was administrated and athird 3D T₁-weighted image (100%-dose) was obtained. Signalnormalization was performed to remove systematic differences (e.g.,transmit and receive gains) that may have caused signal intensitychanges between different acquisitions across different scannerplatforms and hospital sites. Then, nonlinear affine co-registrationbetween pre-dose, 10%-dose, and 100%-dose images were performed. The DLmodel used a U-Net encoder-decoder architecture, with the underlyingassumption that the contrast-related signal between pre-contrast andlow-dose contrast-enhanced images was nonlinearly scaled to thefull-dose contrast images. Images from other contrasts such as T₂ andT₂-FLAIR can be included as part of the input to improve the modelprediction.

As another example of an experiment in connection with FIG. 10 -FIG. 13, data distribution and heterogeneity of a study dataset from threeinstitutions, three different manufacturers, and eight different scannermodels are shown in FIG. 10 . The study retrospectively identified 640patients (323 females; 52±16 years), undergoing clinical brain MRI examsfrom three institutions, three scanner manufacturers and eight scannermodels using different institutional scan protocols, including differentimaging planes, field strengths, voxel sizes, matrix sizes, use of fatsuppression, contrast agents and injection methods. The clinicalindications included suspected tumor, post-op tumor follow-up, routinebrain, and others requiring MRI exams with GBCAs. Each subject underwent3D pre-contrast T₁w imaging, followed by a low-dose contrast-enhancedT₁w scan with 10% (0.01 mmol/kg) of the standard dose (0.1 mmol/kg). Fortraining and evaluation, a third 3D T1w image was obtained with theremaining 90% (0.09 mmol/kg) of the full dose, which was considered asthe ground truth. All three acquisitions were made in a single imagingsession, and the patients did not receive any additional gadolinium dosecompared to the standard protocol.

Out of 640 cases, the model as shown in FIG. 11 was trained with 56cases, and 13 validation cases were used to fine-tune thehyper-parameters and empirically find the optimal combination of lossweights. To ensure that the model generalizes well across sites andvendors, the train and validation sets consisted of approximately anequal number of studies from all the institutions and scannermanufacturers (refer FIG. 10 ). The remaining 571 cases were held-outfor testing and model evaluation. The model was implemented in Python3.5 using Keras with Tensorflow backend and was trained on Nvidia TeslaV100 (SXM2 32 GB) GPU for 100 epochs with a batch size of 8. Modeloptimization was performed using Adam optimizer with a learning rate of0.001.

The model is quantitatively evaluated using a plurality of metrics. Peaksignal-to-noise ratio (PSNR) is the scaled version of pixel-wisedifferences, whereas structural similarity index (SSIM) is sensitive tochanges in local structure and hence captures the structural aspect ofthe predicted image with respect to the ground truth. Using the 571 testcases, the model was quantitatively evaluated using the PSNR and SSIMmetrics, computed between the true full-dose and synthesized images.These values were compared with the PSNR and SSIM values betweenlow-dose and full-dose images. Per-site and per-scanner metrics werealso calculated and compared to prove model generalizability.

From the test set, a subset of images from 26 patients (13 males; 58±15years), with different types and grades of enhancing tumor cases (eitherpre- or post-operative) were identified and used for an in-depthevaluation of model performance. These enhancing tumor cases weresimilar to the training dataset in terms of heterogeneity and wereacquired using the same scanning protocol as shown in FIG. 5 . A binaryassessment was performed to find if the enhancement pattern agreedwithout any false positives or false negatives (with true full-doseimages as the reference). When present, image artifacts in thesynthesized images were recorded and the image artifacts are proved tobe reduced with aid of the provided model.

To further validate that the model predictions were similar to thefull-dose ground truth, automatic tumor segmentation is performed on the26 enhancing tumor cases. The variant of the model applied, used onlypost-contrast images to segment the tumor core. As per the requirementsof the segmentation model, the ground truth and predicted full-doseimages were skull-stripped, interpolated to 1 mm³ resolution andco-registered to an anatomical template. The evaluation is performed bycomputing the Dice scores of the predicted tumor core between thesegmented masks of the ground-truth and those created using thesynthesized images.

FIG. 11 schematically illustrates systems and methods are utilized tomonotonically improve the image quality. The example is shown for asagittally acquired MR image with an enhancing frontal tumor. Verticalstreaks can be seen in the axial reformat of the 2.5D model result asshown in panel a, which was fixed by MPR training and inference as shownin panel b. Adding perceptual and adversarial losses further improvesthe texture inside the tumor and restored overall perceptual quality asshown panel c. Additionally, weighting the L1 loss with the smoothenhancement mask matched the enhancement pattern to that of the groundtruth, as shown in panel d. The monotonic increase in the metrics withrespect to the ground truth (as shown in panel e) also illustrates theimprovement of model. Below table shows the model improvement for eachof the proposed technical solutions for the 26 enhancing tumor cases.

UNet UNet +VGG & +Enhancement Metric 2D (35) 2.5D +MPR GAN mask* PSNR31.84 ± 4.88 32.38 ± 4.67 33.56 ± 5.19 34.28 ± 4.88 35.22 ± 4.79 (dB)SSIM  0.88 ± 0.06  0.89 ± 0.06  0.90 ± 0.06  0.92 ± 0.05  0.93 ± 0.04

FIG. 12 shows pre-contrast, low-dose, full-dose ground truth andsynthesized images along with the quantitative metrics for cases fromdifferent sites and scanners. The metrics show that the model with theproposed technical improvements performed better than the original model(with metrics 31.84±4.88 dB, 0.88±0.06). The best performing model usedMPR with five rotations with a combination of SSIM, perceptual,adversarial, and enhancement weighted L1 losses. For a 512×512×300volume, preprocessing and inference of the best model took about 135seconds on a GeForce RTX 2080 (16 GB) GPU.

FIG. 13 shows examples of different number of rotations and thecorresponding effect on the quality of the image and the performance.The effect of the number of rotation angles in MPR as shown in FIG. 13provides that greater number of angles may reduce the horizontal streaksinside the tumor (better quality), while it may also increase theinference time. When deploy a trained model to a physical site, thenumber of rotations and different angles may be determined based on thedesired image quality and deployment environment (e.g., computationalpower, memory storage, etc.).

System Overview

The provided DL framework for low-dose contrast-enhanced MRI is capableof reducing the dosage of GBCA for contrast-enhanced MRI whilepreserving image quality and avoiding degradation in contrastenhancement. The robustness and generalizability of the DL model isimproved thereby allowing for improved adaptation to variousapplications across a heterogeneous patient and site population. FIG. 4schematically illustrates a magnetic resonance imaging (MRI) system 400in which an imaging enhancer 440 of the presenting disclosure may beimplemented. The MRI system 400 may comprise a magnet system 403, apatient transport table 405 connected to the magnet system, and acontroller 401 operably coupled to the magnet system. In one example, apatient may lie on the patient transport table 405 and the magnet system403 would pass around the patient. The controller 401 may controlmagnetic fields and radio frequency (RF) signals provided by the magnetsystem 403 and may receive signals from detectors in the magnet system403.

The MRI system 400 may further comprise a computer system 410 and one ormore databases operably coupled to the controller 401 over the network430. The computer system 410 may be used for implementing the volumetricMR imaging enhancer 440. The volumetric MR imaging enhancer 440 mayimplement the DL framework and methods described herein. For example,the volumetric MR imaging enhancer may employ the MPR reconstructionmethod and various other training algorithms, and data processingmethods described herein. The computer system 410 may be used forgenerating an imaging enhancer using training datasets. Although theillustrated diagram shows the controller and computer system as separatecomponents, the controller and computer system can be integrated into asingle component.

The computer system 410 may comprise a laptop computer, a desktopcomputer, a central server, distributed computing system, etc. Theprocessor may be a hardware processor such as a central processing unit(CPU), a graphic processing unit (GPU), a general-purpose processingunit, which can be a single core or multi core processor, or a pluralityof processors for parallel processing. The processor can be any suitableintegrated circuits, such as computing platforms or microprocessors,logic devices and the like. Although the disclosure is described withreference to a processor, other types of integrated circuits and logicdevices are also applicable. The processors or machines may not belimited by the data operation capabilities. The processors or machinesmay perform 512 bit, 256 bit, 128 bit, 64 bit, 32 bit, or 16 bit dataoperations.

The MRI system 400 may include one or more databases 420 that mayutilize any suitable database techniques. For instance, structured querylanguage (SQL) or “NoSQL” database may be utilized for storing thereconstructed/reformat image data, raw collected data, trainingdatasets, trained model (e.g., hyper parameters), weightingcoefficients, rotation angles, rotation numbers, orientation forreformat reconstruction, etc. Some of the databases may be implementedusing various standard data-structures, such as an array, hash, (linked)list, struct, structured text file (e.g., XML), table, JSON, NOSQLand/or the like. Such data-structures may be stored in memory and/or in(structured) files. In another alternative, an object-oriented databasemay be used. Object databases can include a number of object collectionsthat are grouped and/or linked together by common attributes; they maybe related to other object collections by some common attributes.Object-oriented databases perform similarly to relational databases withthe exception that objects are not just pieces of data but may haveother types of functionality encapsulated within a given object. If thedatabase of the present disclosure is implemented as a data-structure,the use of the database of the present disclosure may be integrated intoanother component such as the component of the present invention. Also,the database may be implemented as a mix of data structures, objects,and relational structures. Databases may be consolidated and/ordistributed in variations through standard data processing techniques.Portions of databases, e.g., tables, may be exported and/or imported andthus decentralized and/or integrated.

The network 430 may establish connections among the components in theMRI platform and a connection of the MRI system to external systems. Thenetwork 430 may comprise any combination of local area and/or wide areanetworks using both wireless and/or wired communication systems. Forexample, the network 430 may include the Internet, as well as mobiletelephone networks. In one embodiment, the network 430 uses standardcommunications technologies and/or protocols. Hence, the network 430 mayinclude links using technologies such as Ethernet, 802.11, worldwideinteroperability for microwave access (WiMAX), 2G/3G/4G/5G mobilecommunications protocols, InfiniBand, PCI Express Advanced Switching,etc. Other networking protocols used on the network 430 can includemultiprotocol label switching (MPLS), the transmission controlprotocol/Internet protocol (TCP/IP), the User Datagram Protocol (UDP),the hypertext transport protocol (HTTP), the simple mail transferprotocol (SMTP), the file transfer protocol (FTP), and the like. Thedata exchanged over the network can be represented using technologiesand/or formats including image data in binary form (e.g., PortableNetworks Graphics (PNG)), the hypertext markup language (HTML), theextensible markup language (XML), etc. In addition, all or some of linkscan be encrypted using conventional encryption technologies such assecure sockets layers (SSL), transport layer security (TLS), InternetProtocol security (IPsec), etc. In another embodiment, the entities onthe network can use custom and/or dedicated data communicationstechnologies instead of, or in addition to, the ones described above.

Whenever the term “at least,” “greater than,” or “greater than or equalto” precedes the first numerical value in a series of two or morenumerical values, the term “at least,” “greater than” or “greater thanor equal to” applies to each of the numerical values in that series ofnumerical values. For example, greater than or equal to 1, 2, or 3 isequivalent to greater than or equal to 1, greater than or equal to 2, orgreater than or equal to 3.

Whenever the term “no more than,” “less than,” or “less than or equalto” precedes the first numerical value in a series of two or morenumerical values, the term “no more than,” “less than,” or “less than orequal to” applies to each of the numerical values in that series ofnumerical values. For example, less than or equal to 3, 2, or 1 isequivalent to less than or equal to 3, less than or equal to 2, or lessthan or equal to 1.

As used herein A and/or B encompasses one or more of A or B, andcombinations thereof such as A and B. It will be understood thatalthough the terms “first,” “second,” “third” etc. are used herein todescribe various elements, components, regions and/or sections, theseelements, components, regions and/or sections should not be limited bythese terms. These terms are merely used to distinguish one element,component, region or section from another element, component, region orsection. Thus, a first element, component, region or section discussedherein could be termed a second element, component, region or sectionwithout departing from the teachings of the present invention.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” or “includes” and/or “including,” when used in thisspecification, specify the presence of stated features, regions,integers, steps, operations, elements and/or components, but do notpreclude the presence or addition of one or more other features,regions, integers, steps, operations, elements, components and/or groupsthereof.

Reference throughout this specification to “some embodiments,” or “anembodiment,” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment. Thus, the appearances of the phrase “in someembodiment,” or “in an embodiment,” in various places throughout thisspecification are not necessarily all referring to the same embodiment.Furthermore, the particular features, structures, or characteristics maybe combined in any suitable manner in one or more embodiments

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. It is not intendedthat the invention be limited by the specific examples provided withinthe specification. While the invention has been described with referenceto the aforementioned specification, the descriptions and illustrationsof the embodiments herein are not meant to be construed in a limitingsense. Numerous variations, changes, and substitutions will now occur tothose skilled in the art without departing from the invention.Furthermore, it shall be understood that all aspects of the inventionare not limited to the specific depictions, configurations or relativeproportions set forth herein which depend upon a variety of conditionsand variables. It should be understood that various alternatives to theembodiments of the invention described herein may be employed inpracticing the invention. It is therefore contemplated that theinvention shall also cover any such alternatives, modifications,variations or equivalents. It is intended that the following claimsdefine the scope of the invention and that methods and structures withinthe scope of these claims and their equivalents be covered thereby.

1. (canceled)
 2. A computer-implemented method for improving imagequality of medical image, the method comprising: receiving a volumetricmedical image of a subject; generating one or more reformat volumetricmedical images by reformatting the volumetric medical image of thesubject in one or more orientations; and feeding an input comprising theone or more reformat volumetric medical images to a deep network modelto generate a predicted medical image with improved quality.
 3. Thecomputer-implemented method of claim 2, wherein the volumetric medicalimage is acquired in a first direction of a scanning plane.
 4. Thecomputer-implemented method of claim 3, wherein the one or moreorientations comprise a second direction that is different from thefirst direction of the scanning plane.
 5. The computer-implementedmethod of claim 2, wherein the volumetric medical image is acquiredusing a transforming magnetic resonance (MR) device.
 6. Thecomputer-implemented method of claim 2, wherein the volumetric medicalimage is a 2.5D volumetric image.
 7. The computer-implemented method ofclaim 2, wherein the volumetric medical image is created by stacking anumber of image slices channel-wise.
 8. The computer-implemented methodof claim 2, further comprising rotating the one or more reformatvolumetric medical images into various angles to generate one or morerotated reformat medical images.
 9. The computer-implemented method ofclaim 7, further comprising applying the deep network model to the oneor more rotated reformat medical images to output one or more predictedimages.
 10. The computer-implemented method of claim 2, wherein the oneor more predicted images are rotated to be aligned to a scanning plane.11. The computer-implemented method of claim 2, wherein the deep networkmodel is a 2.5 D trained model.
 12. A non-transitory computer-readablestorage medium including instructions that, when executed by one or moreprocessors, cause the one or more processors to perform operationscomprising: receiving a volumetric medical image of a subject;generating one or more reformat volumetric medical images byreformatting the volumetric medical image of the subject in one or moreorientations; and feeding an input comprising the one or more reformatvolumetric medical images to a deep network model to generate apredicted medical image with improved quality.
 13. The non-transitorycomputer-readable storage medium of claim 12, wherein the volumetricmedical image is acquired in a first direction of a scanning plane. 14.The non-transitory computer-readable storage medium of claim 13, whereinthe one or more orientations comprise a second direction that isdifferent from the first direction of the scanning plane.
 15. Thenon-transitory computer-readable storage medium of claim 12, wherein thevolumetric medical image is acquired using a transforming magneticresonance (MR) device.
 16. The non-transitory computer-readable storagemedium of claim 12, wherein the volumetric medical image is a 2.5Dvolumetric image.
 17. The non-transitory computer-readable storagemedium of claim 12, wherein the volumetric medical image is created bystacking a number of image slices channel-wise.
 18. The non-transitorycomputer-readable storage medium of claim 12, wherein the operationsfurther comprise rotating the one or more reformat volumetric medicalimages into various angles to generate one or more rotated reformatmedical images.
 19. The non-transitory computer-readable storage mediumof claim 12, wherein the operations further comprise applying the deepnetwork model to the one or more rotated reformat medical images tooutput one or more predicted images.
 20. The non-transitorycomputer-readable storage medium of claim 12, wherein the one or morepredicted images are rotated to be aligned to a scanning plane.
 21. Thenon-transitory computer-readable storage medium of claim 12, wherein thedeep network model is a 2.5 D trained model.